Identifying table tennis balls from real match scenes using image processing and artificial intelligence techniques

Table tennis is a fast sport and it is very difficult for a normal human being to manage accurate umpiring, especially in services (serves), which usually take less than a second to complete. The umpire needs to make over 30 observations and makes a judgment before or soon after the service is complete. This is a complex task and the author believes the employment of image processing and artificial intelligence (AI) technologies could aid the umpire to evaluating services more accurately. The aim of this research is to develop an intelligent system which is able to identify and track the location of the ball from live video images and evaluate the service according to the service rules. In this paper, the discussion is focused on the development of techniques for identifying a table tennis ball from match scenes. These techniques formed the basis of the ball detection system. Artificial neural networks (ANN) have been designed and applied to further the accuracy of the detection system. The system has been tested on still images taken at real match scenes and the preliminary results are very promising. Almost all the balls from the images have been correctly identified. The system has been further tested on some video images and the preliminary result is also very encouraging. It shows the system could tolerate the poorer quality of video images. This paper also discusses the idea of employing multiple cameras for improving accuracy. A multi-agent system is proposed because it is known to be able to coordinate and manage the flow of information more effectively

Multiview System for Tracking a Fast Moving Object Against Complex Backgrounds

Tracking the real world coordinate of a fast moving object against a complex background is very challenging. When designing a multi-view system for this purpose, one key consideration is the arrangement of the cameras such that the object can be constantly and accurately tracked. This paper discusses a novel cameras arrangement, which can provide redundancy for fault tolerance, yet do not require installing more cameras nor relying aerial views of the scene. Using a table tennis match as example, experiment results show that the multi-view system with this cameras arrangement has a promising potential for tracking a table tennis ball in a real match scene

Developing an intelligent table tennis umpiring system

The aim of this research is to develop an intelligent system which is able to evaluate table tennis services independently and aid the umpire to make a more accurate decision. Table tennis is a fast sport. A service usually takes one to a few seconds to complete but there are many observations an umpire needs to take and makes a judgment before or soon after the service is complete. This is a complex and time-critical task and the author believes the employment of videography, image processing and artificial intelligence (AI) technologies could help evaluating the service. This paper outlines the idea and the plan of developing an intelligent assistant for table tennis umpire in evaluating services. The system is expected to be able to track the location of the ball from live video images and evaluate the service according to the service rules. In this pilot study, however, the focus is on the development of the techniques, rather than building a complete system. Various videography, image processing and artificial intelligence techniques will be experimented and evaluated. A prototype system is to be built. The system will be compared and tested against the judgements of a human umpire. Both the accuracy and rate of responses will be concerned. The ultimate goal is to further develop the system so that it can umpire and score table tennis matches autonomously. The system may also benefit players who want to have their services evaluated in real time without the need of having a human umpire present

Developing an Intelligent Table Tennis Umpiring System: Identifying the ball from the scene

This paper reports further development of an intelligent table tennis umpiring system, of which the idea and plan was previously published at this conference in 2007. Briefly, table tennis is a fast sport. A service usually takes a few seconds to complete but an umpire needs to make many observations and makes a judgment before or soon after the service is complete. This is a complex task and the author believes the employment of videography, image processing and artificial intelligence (AI) technologies could help evaluating the service. The aim of this research is to develop an intelligent system which is able to identify and track the location of the ball from live video images and evaluate the service according to the service rules. In this paper, the techniques of identifying a table tennis ball from the scene is described and discussed. A number of image processing techniques have been employed to identify and measure the characteristics of the ball. Artificial neural networks have been applied as a classifier. It classifies whether the detected object is not-a- ball, a ball on the palm or a ball in mid air. The system has been tested on 21 still images which contain pictures of ball-like objects, balls on the palm and in mid air. The preliminary results are very promising. Out of 83 objects, 82 have been correctly classified. The system will be further tested on video images once the video is captured and processed. This paper also discusses the idea of implementing the final system as a multi-agent system, which the author believes it is appropriate for this application because multiple cameras will have to be employed to obtain accurate results

Tracking Table Tennis Balls in Real Match Scenes for Umpiring Applications

Judging the legitimacy of table tennis services presents many challenges where technology can be judiciously applied to enhance decision-making. This paper presents a purpose-built system to automatically detect and track the ball during table-tennis services to enable precise judgment over their legitimacy in real-time. The system comprises a suite of algorithms which adaptively exploit spatial and temporal information from real match video sequences, which are generally characterised by high object motion, allied with object blurring and occlusion. Experimental results on a diverse set of table-tennis test sequences corroborate the system performance in facilitating consistently accurate and efficient decision-making over the validity of a service

Transform domain distributed video coding using larger transform blocks

Distributed Video Coding (DVC) displays promising performance at low spatial resolutions but begins to struggle as the resolution increases. One of the limiting aspects is its 4x4 block size of Discrete Cosine Transform (DCT) which is often impractical at higher resolutions. This paper investigates the impact of exploiting larger DCT block sizes on the performance of transform domain DVC at higher spatial resolutions. In order to utilize a larger block size in DVC, appropriate quantisers have to be selected and this has been solved by means of incorporating a content-aware quantisation mechanism to generate image specific quantisation matrix for any DCT block size. Experimental results confirm that the larger 8x8 block size consistently exhibit superior RD performance for CIF resolution sequences compared to the smaller 4x4 block sizes. Significant PSNR improvement has been observed for 16x16 block size at 4CIF resolution with up to 1.78dB average PSNR gain compared to its smaller block alternatives

Wyner-Ziv side information generation using a higher order piecewise trajectory temporal interpolation algorithm

Distributed video coding (DVC) reverses the traditional coding paradigm of complex encoders allied with basic decoding, to one where the computational cost is largely incurred by the decoder. This enables low-cost, resource-poor sensors to be used at the transmitter in various applications including multi-sensor surveillance. A key constraint governing DVC performance is the quality of side information (SI), a coarse representation of original video frames which are not available at the decoder. Techniques to generate SI have generally been based on linear temporal interpolation, though these do not always produce satisfactory SI quality especially in sequences exhibiting asymmetric (non-linear) motion. This paper presents a higher-order piecewise trajectory temporal interpolation (HOPTTI) algorithm for SI generation that quantitatively and perceptually affords better SI quality in comparison to existing temporal interpolation-based approaches

A content-aware quantisation mechanism for transform domain distributed video coding

The discrete cosine transform (DCT) is widely applied in modern codecs to remove spatial redundancies, with the resulting DCT coefficients being quantised to achieve compression as well as bit-rate control. In distributed video coding (DVC) architectures like DISCOVER, DCT coefficient quantisation is traditionally performed using predetermined quantisation matrices (QM), which means the compression is heavily dependent on the sequence being coded. This makes bit-rate control challenging, with the situation exacerbated in the coding of high resolution sequences due to QM scarcity and the non-uniform bit-rate gaps between them. This paper introduces a novel content-aware quantisation (CAQ) mechanism to overcome the limitations of existing quantisation methods in transform domain DVC. CAQ creates a frame-specific QM to reduce quantisation errors by analysing the distribution of DCT coefficients. In contrast to the predetermined QM that is applicable to only 4x4 block sizes, CAQ produces QM for larger block sizes to enhance compression at higher resolutions. This provides superior bit-rate control and better output quality by seeking to fully exploit the available bandwidth, which is especially beneficial in bandwidth constrained scenarios. In addition, CAQ generates superior perceptual results by innovatively applying different weightings to the DCT coefficients to reflect the human visual system. Experimental results corroborate that CAQ both quantitatively and qualitatively provides enhanced output quality in bandwidth limited scenarios, by consistently utilising over 90% of available bandwidth

Tracking a table tennis ball for umpiring purposes using a multi-agent system

Tracking a table tennis ball for umpiring purposes is a challenging task as, in real-match scenarios, the ball travels fast and can become occluded or merged with other background objects. This paper presents the design of a multi-view based tracking system that can overcome the challenges of tracking a ball in real match sequences. The system has been tested on a complete table tennis rally and the results are very promising. The system is able to continuously track the ball with only marginal variations in detection. Furthermore, the initialization of the multi-camera system means it is both a portable and cost-effective solution for umpiring purposes

Tracking a table tennis ball for umpiring purposes

This study investigates tracking a table-tennis ball rapidly from video captured using low-cost equipment for umpiring purposes. A number of highly efficient algorithms have been developed for this purpose. The proposed system was tested using sequences capture from real match scenes. The preliminary results of experiments show that accurate and rapid tracking can be achieved even under challenging conditions, including occlusion and colour merging. This work can contribute to the development of an automatic umpiring system and also has the potential to provide amateur users open access to a detection tool for fast-moving, small, round objects